Autonomous Wall-climbing Robots for Inspection and Maintenance of Concrete Bridges

Since 2002, the PI’s group has developed four generations of wall-climbing robots for NDE inspection of civil infrastructure. These robots combine the advantages of aerodynamic attraction and suction to achieve a desirable balance of strong adhesion and high mobility. They don’t require perfect sealing and can thus move on smooth and rough surfaces, such as brick, concrete, stucco, wood, glass, and metal. For example, Rise-Rover uses two drive modules to carry their middle compartment with payload up to 450 N. Ground penetrating radar (GPR)-Rover and Mini GPR-Rover are custom designed to carry a GSSI’s GPR antenna for subsurface defect detection and utility survey on concrete structures such as bridges and tunnels. The robots can also carry other devices such as impact echo and ultrasonic flaw detectors for bridge evaluation. To date, all the robots are remotely controlled to scan concrete surfaces. This project aims to develop motion control and localization methods to make wall-climbing robots a fully autonomous system with automated inspection process using various NDE devices and sensors, and design innovative mechanisms and tools and integrate them into the robots for maintenance actions

Robotic Inspection of Infrastructure Using Vision, GPR, and Impact-Echo Sensors

Bridges, dams, highways, and tunnels in the U.S. are reaching their life expectancy, and thus have imperative needs for routine inspection and maintenance to ensure sustainability. It is reported that 42% of over 600,000 highway bridges in the National Bridge Inventory (NBI) have exceeded their design life of 50 years, and 42,951 bridges are rated in poor condition and classified as structurally deficient . To inspect the structurally integrity of bridges, the inspectors also need to detect subsurface defects (i.e., delamination, voids) using NDE instruments such as GPR and impact-echo (IE) device at difficult to access components (i.e., pier, bottom side of the deck). The current practice of manual inspection hand-held NDE devices by a spider-man with rope access, or by using scaffolding or by using snooper truck has to block traffic, and is expensive, time-consuming, and exposes human inspectors to dangerous situations. This presentation will introduce climbing robots developed over the years at CCNY Robotics Lab that integrate the robot control and vision-based accurate positioning with NDE signal processing to detect both surface flaws and subsurface defects. The use of the robotic inspection tool will eliminate the time, hassle, and cost to layout grid lines on flat terrain, and make it possible to automatically collect NDE data with minimum human intervention. This presentation will also introduce machine learning algorithms for visual inspection to detect and measure cracks: IE data processing methods that utilizes both learning-based and classical methods to interpret the IE data and reveal subsurface objects; and DNN-based GPR data analysis software to reveal subsurface targets for better visualization

Toward Autonomous Wall-Climbing Robots for Inspection of Concrete Bridges and Tunnels

In addition to visual inspection for surface flaws, inspectors are often required to detect subsurface defects (e.g., delamination and voids) using nondestructive evaluation (NDE) instruments, such as ground penetration radar (GPR) and impact sounding device, in order to determine the structural integrity of bridges and tunnels. In these cases, access to critical locations for reliable and safe inspections is a challenge. Since 2002, Dr. Jizhong Xiao’s group has developed four generations of wall-climbing robots for NDE inspection of bridges and tunnels. These robots combine the advantages of aerodynamic attraction and suction to achieve a desirable balance of strong adhesion and high mobility. For example, Rise-Rover with two drive modules can carry up to 450 N payload, and GPR-Rover can carry a small GPR antenna for subsurface flaw detection and utility survey on concrete structures. These robots can reach difficult-to-access areas (e.g., the bottom side of bridge decks), take close-up pictures, record and transmit NDE data to a host computer for further analysis. They can potentially make bridge inspection faster, safer, and cheaper without affecting traffic flow on roadways. This presentation will review the recent development of smart and autonomous wall-climbing robots to realize automated inspection of civil infrastructure with minimal human intervention

Ego-Downward and Ambient Video based Person Location Association

Using an ego-centric camera to do localization and tracking is highly needed for urban navigation and indoor assistive system when GPS is not available or not accurate enough. The traditional hand-designed feature tracking and estimation approach would fail without visible features. Recently, there are several works exploring to use context features to do localization. However, all of these suffer severe accuracy loss if given no visual context information. To provide a possible solution to this problem, this paper proposes a camera system with both ego-downward and third-static view to perform localization and tracking in a learning approach. Besides, we also proposed a novel action and motion verification model for cross-view verification and localization. We performed comparative experiments based on our collected dataset which considers the same dressing, gender, and background diversity. Results indicate that the proposed model can achieve $18.32 \%$ improvement in accuracy performance. Eventually, we tested the model on multi-people scenarios and obtained an average $67.767 \%$ accuracy

Deep Semantic 3D Visual Metric Reconstruction Using Wall-Climbing Robot

This project introduces an inspection method using a deep neural network to detect the crack and spalling defects on concrete structures performed by a wall-climbing robot. First, we create a pixel-level semantic dataset which includes 820 labeled images. Second, we propose an inspection method to obtain 3D metric measurement by using an RGB-D camera-based visual simultaneous localization and mapping (SLAM), which is able to generate pose coupled key-frames with depth information. Therefore, the semantic inspection results can be registered in the concrete structure 3D model for condition assessment and monitoring. Third, we present our new generation wall-climbing robot to perform the inspection task on both horizontal and vertical surfaces

A Random Multi-Trajectory Generation Method for Online Emergency Threat Management (Analysis and Application in Path Planning Algorithm)

This paper presents a novel randomized path planning algorithm, which is a goal and homology biased sampling based algorithm called Multiple Guiding Attraction based Random Tree, and robots can use it to tackle pop-up and moving threats under kinodynamic constraints. Our proposed method considers the kinematics and dynamics constraints, using obstacle information to perform informed sampling and redistribution around collision region toward valid routing. We pioneeringly propose a multiple path planning method using ‘Extending Forbidden’ algorithm, rather than using variant cost principles for online threat management. The threat management method performs online path switching between the planned multiple paths, which is proved with better time performance than conventional approaches. The proposed method has advantage in exploration in obstacle crowded environment, where narrow corridor fails using the general sampling based exploration methods. We perform detailed comparative experiments with peer approaches in cluttered environment, and point out the advantages in time and mission performance

Design and Analysis of a Single-Camera Omnistereo Sensor for Quadrotor Micro Aerial Vehicles (MAVs)

We describe the design and 3D sensing performance of an omnidirectional stereo (omnistereo) vision system applied to Micro Aerial Vehicles (MAVs). The proposed omnistereo sensor employs a monocular camera that is co-axially aligned with a pair of hyperboloidal mirrors (a vertically-folded catadioptric configuration). We show that this arrangement provides a compact solution for omnidirectional 3D perception while mounted on top of propeller-based MAVs (not capable of large payloads). The theoretical single viewpoint (SVP) constraint helps us derive analytical solutions for the sensor’s projective geometry and generate SVP-compliant panoramic images to compute 3D information from stereo correspondences (in a truly synchronous fashion). We perform an extensive analysis on various system characteristics such as its size, catadioptric spatial resolution, field-of-view. In addition, we pose a probabilistic model for the uncertainty estimation of 3D information from triangulation of back-projected rays. We validate the projection error of the design using both synthetic and real-life images against ground-truth data. Qualitatively, we show 3D point clouds (dense and sparse) resulting out of a single image captured from a real-life experiment. We expect the reproducibility of our sensor as its model parameters can be optimized to satisfy other catadioptric-based omnistereo vision under different circumstances